Feature Selection and Classifier Development for Radio Frequency Device Identification

The proliferation of simple and low-cost devices, such as IEEE 802.15.4 ZigBee and Z-Wave, in Critical Infrastructure (CI) increases security concerns. Radio Frequency Distinct Native Attribute (RF-DNA) Fingerprinting facilitates biometric-like identification of electronic devices emissions from variances in device hardware. Developing reliable classifier models using RF-DNA fingerprints is thus important for device discrimination to enable reliable Device Classification (a one-to-many looks most like assessment) and Device ID Verification (a one-to-one looks how much like assessment). AFITs prior RF-DNA work focused on Multiple Discriminant Analysis/Maximum Likelihood (MDA/ML) and Generalized Relevance Learning Vector Quantized Improved (GRLVQI) classifiers. This work 1) introduces a new GRLVQI-Distance (GRLVQI-D) classifier that extends prior GRLVQI work by supporting alternative distance measures, 2) formalizes a framework for selecting competing distance measures for GRLVQI-D, 3) introducing response surface methods for optimizing GRLVQI and GRLVQI-D algorithm settings, 4) develops an MDA-based Loadings Fusion (MLF) Dimensional Reduction Analysis (DRA) method for improved classifier-based feature selection, 5) introduces the F-test as a DRA method for RF-DNA fingerprints, 6) provides a phenomenological understanding of test statistics and p-values, with KS-test and F-test statistic values being superior to p-values for DRA, and 7) introduces quantitative dimensionality assessment methods for DRA subset selection

Analytics for Autonomous C4ISR within e-Government: a Research Agenda

e-Government enables big data analytics to support decision processes in governing. C4ISR (Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance) is essentially e-Government scoped to military decision processes. The value of big data and its challenges are common to both. High variety and demand for veracity compel domain expertise-specific data analysis, and increasing volume and velocity hinder data analytics at scale. These conditions challenge even highly automated methods for comprehensive cross-domain analytics, and motivate cognitive approaches such as underlie Autonomous Systems (AS) aimed at C4ISR. A C4ISR framework is examined by parts, linking each C to ISR capability, and a taxonomy of analytics is extended to include cognitive autonomy enablers. Coupling these frameworks, the authors propose an extension of cognitive approaches for autonomy in C4ISR to e-Government in general and outline a research agenda for attaining it

Multivariate Stochastic Approximation to Tune Neural Network Hyperparameters for Criticial Infrastructure Communication Device Identification

The e-government includes Wireless Personal Area Network (WPAN) enabled internet-to-government pathways. Of interest herein is Z-Wave, an insecure, low-power/cost WPAN technology increasingly used in critical infrastructure. Radio Frequency (RF) Fingerprinting can augment WPAN security by a biometric-like process that computes statistical features from signal responses to 1) develop an authorized device library, 2) develop classifier models and 3) vet claimed identities. For classification, the neural network-based Generalized Relevance Learning Vector Quantization-Improved (GRLVQI) classifier is employed. GRLVQI has shown high fidelity in classifying Z-Wave RF Fingerprints; however, GRLVQI has multiple hyperparameters. Prior work optimized GRLVQI via a full factorial experimental design. Herein, optimizing GRLVQI via stochastic approximation, which operates by iterative searching for optimality, is of interest to provide an unconstrained optimization approach to avoid limitations found in full factorial experimental designs. The results provide an improvement in GRLVQI operation and accuracy. The methodology is further generalizable to other problems and algorithms

Clustering and Topological Data Analysis: Comparison and Application

Clustering is common technique used to demonstrate relationships between data and information. Of recent interest is topological data analysis (TDA), which can represent and cluster data through persistent homology. The TDA algorithms used include the Topological Mode Analysis Tool (ToMATo) algorithm, Garin and Tauzin’s TDA Pipeline, and the Mapper algorithm. First, TDA is compared to ten other clustering algorithms on artificial 2D data where it ranked third overall. TDA had the second-highest performance in terms of average accuracy (97.9%); however, its computation-time performance ranked in the middle of the algorithms. TDA ranked fourth on the qualitative “visual trustworthiness” metric. On real-world data, TDA showed promising classification results (accuracy between 80-95%). Overall, this paper shows TDA is a competitive algorithm performance-wise, though computationally expensive. When TDA is used for visualization, the Mapper algorithm allows for unique alternative views especially effective for visualizing highly dimensional data

An Optimization Framework for Generalized Relevance Learning Vector Quantization with Application to Z-Wave Device Fingerprinting

Z-Wave is low-power, low-cost Wireless Personal Area Network (WPAN) technology supporting Critical Infrastructure (CI) systems that are interconnected by government-to-internet pathways. Given that Z-wave is a relatively unsecure technology, Radio Frequency Distinct Native Attribute (RF-DNA) Fingerprinting is considered here to augment security by exploiting statistical features from selected signal responses. Related RF-DNA efforts include use of Multiple Discriminant Analysis (MDA) and Generalized Relevance Learning Vector Quantization-Improved (GRLVQI) classifiers, with GRLVQI outperforming MDA using empirically determined parameters. GRLVQI is optimized here for Z-Wave using a full factorial experiment with spreadsheet search and response surface methods. Two optimization measures are developed for assessing Z-Wave discrimination: 1) Relative Accuracy Percentage (RAP) for device classification, and 2) Mean Area Under the Curve (AUCM) for device identity (ID) verification. Primary benefits of the approach include: 1) generalizability to other wireless device technologies, and 2) improvement in GRLVQI device classification and device ID verification performance

Integration of Computer Vision with Analogical Reasoning for Characterizing Unknowns

Current state-of-the-art artificial intelligence struggles with accurate interpretation of out-of-library (OOL) objects. One method proposed remedy is analogical reasoning (AR), which utilizes abductive reasoning to draw inferences on an unfamiliar scenario given knowledge about a similar familiar scenario. Currently, applications of visual AR gravitate toward analogy-formatted image problems rather than to computer vision data sets. The Image Recognition Through Analogical Reasoning Algorithm (IRTARA) approach described herein shows how AR can be leveraged to improve computer vision in OOL situations. IRTARA produces a word-based term frequency list that characterizes the OOL object of interest. To evaluate the quality of the results of IRTARA, both quantitative and qualitative assessments are used, including a baseline to compare the automated methods with human-generated results. Fifteen OOL objects were tested using IRTARA, which showed consistent results across all three evaluation methods on the objects that performed exceptionally well or poorly overall

Easy and Efficient Hyperparameter Optimization to Address Some Artificial Intelligence “ilities”

Artificial Intelligence (AI), has many benefits, including the ability to find complex patterns, automation, and meaning making. Through these benefits, AI has revolutionized image processing among numerous other disciplines. AI further has the potential to revolutionize other domains; however, this will not happen until we can address the “ilities”: repeatability, explain-ability, reliability, use-ability, trust-ability, etc. Notably, many problems with the “ilities” are due to the artistic nature of AI algorithm development, especially hyperparameter determination. AI algorithms are often crafted products with the hyperparameters learned experientially. As such, when applying the same algorithm to new problems, the algorithm may not perform due to inappropriate settings. This research aims to provide a straightforward and reliable approach to automatically determining suitable hyperparameter settings when given an AI algorithm. Results, show reasonable performance is possible and end-to-end examples are given for three deep learning algorithms and three different data problems

Autonomous Search and Rescue with Modeling and Simulation and Metrics

Unmanned Aerial Vehicles (UAVs) provide rapid exploration capabilities in search and rescue missions while accepting more risks than human operations. One limitation in that current UAVs are heavily manpower intensive and such manpower demands limit abilities to expand UAV use. In operation, manpower demands in UAVs range from determining tasks, selecting waypoints, manually controlling platforms and sensors, and tasks in between. Often, even a high level of autonomy is possible with human generated objectives and then autonomous resource allocation, routing, and planning. However, manually generating tasks and scenarios is still manpower intensive. To reduce manpower demands and move towards more autonomous operations, the authors develop an adaptive planning system that takes high level goals from a human operator and translates them into situationally relevant tasking. For expository simulation, the authors further describe constructing a scenario around the 2018 Hawaii Puna lava natural disaster

Analogical Reasoning: An Algorithm Comparison for Natural Language Processing

There is a continual push to make Artificial Intelligence (AI) as human-like as possible; however, this is a difficult task. A significant limitation is the inability of AI to learn beyond its current comprehension. Analogical reasoning (AR), whereby learning by analogy occurs, has been proposed as one method to achieve this goal. Current AR models have their roots in symbolist, connectionist, or hybrid approaches which indicate how analogies are evaluated. No current studies have compared psychologically-inspired and natural language processing (NLP)-produced algorithms to one another; this study compares seven AR algorithms from both realms on multiple-choice word-based analogy problems. Assessment is based on selection of the correct answer, “correctness,” and their similarity score prediction compared to the “ideal” score, which is defined as the “goodness” metric. Psychologically-based models have an advantage based on our metrics; however, there is not a clear one-size-fits-all algorithm for all AR problems

Bayesian Augmentation of Deep Learning to Improve Video Classification

Traditional automated video classification methods lack measures of uncertainty, meaning the network is unable to identify those cases in which its predictions are made with significant uncertainty. This leads to misclassification, as the traditional network classifies each observation with same amount of certainty, no matter what the observation is. Bayesian neural networks are a remedy to this issue by leveraging Bayesian inference to construct uncertainty measures for each prediction. Because exact Bayesian inference is typically intractable due to the large number of parameters in a neural network, Bayesian inference is approximated by utilizing dropout in a convolutional neural network. This research compared a traditional video classification neural network to its Bayesian equivalent based on performance and capabilities. The Bayesian network achieves higher accuracy than a comparable non-Bayesian video network and it further provides uncertainty measures for each classification